Content data indexing with content associations

ABSTRACT

A full text indexing system is provided for processing content associated with data applications such as encyclopedia and dictionary applications. A build process collects data from various sources, processes the data into constituent parts, including alternative word sets, and stores the constituent parts in structured database tables. A run-time process is used to query the database tables and the results in order to provide effective matches in an efficient manner. Run-time processing is optimized by preprocessing all steps that are query-independent during the build process. A double word table representing all possible word pair combinations for each index entry and an alternative word table are used to further optimize runtime processing.

RELATED PATENT APPLICATIONS

This patent application is a continuation of co-pending, non-provisionalU.S. patent application Ser. No. 10/187,859, entitled “CONTENT DATAINDEXING,” and filed on Jul. 1, 2002. This patent application is alsorelated to non-provisional, co-pending U.S. application Ser. No. ______,entitled “CONTENT DATA INDEXING AND RESULT RANKING” filed concurrentlyherewith, to U.S. application Ser. No. 09/867,228, entitled “METHOD ANDSYSTEM FOR SEARCHING INDEX DATABASES”, which issued as U.S. Pat. No.6,775,666 on Aug. 10, 2004, and to U.S. application Ser. No. 10/335,654,entitled “DATABASE BUILD FOR WEB DELIVERY”, which issued as U.S. Pat.No. 6,983,287 on Jan. 3, 2006. Each of the above also assigned toMicrosoft Corporation and is expressly hereby incorporated by referencein their entireties.

FIELD OF THE INVENTION

The present invention relates to searching content data and morespecifically relates to the indexing of content data in a build processto optimize search speed and efficacy during a run-time process.

BACKGROUND OF THE INVENTION

In response to the development of computers that can processincreasingly larger amounts of data, encyclopedias, dictionaries, andother content data applications have been implemented in electronicform. Such content data applications make it possible to compile andmake available vast amounts of information. However to be useful, thedata must be searchable. More recent developments include theimplementation of such data applications in a network environment, suchas over the Internet. Typically, network implementations can requiresignificant system resources (e.g., computer memory and processor time)to effectively process search queries.

One example of a data content application is the “ENCARTA” brandMultimedia Encyclopedia Application developed and marketed by MicrosoftCorporation of Redmond, Wash. The “ENCARTA” brand MultimediaEncyclopedia Application can be run as a stand-alone application on anindividual computer or can be operated over a network, such as theInternet. Electronic encyclopedias typically have a massive content datavolume that includes all of the articles and other media necessary torender an electronic version of an encyclopedia.

However, to be efficiently used data content applications must be ableto process search queries effectively and quickly. As the amount ofcontent increases, the need for more speed increases. Various prior artsystems have been developed to speed up content data searching. One ofthe most common methods of speeding data searching is to use partialdata searching. This method speeds data searching by designating only asubset of the entire body of data as searchable. Another known method isto associate searchable key words with an un-searchable body of textdata, whereby a search query is processed only against the key words anda match results in returning a reference to the un-searchable body oftext data. Neither of these methods is completely satisfactory, becauseit is impossible to fully predict what search terms a user will selectto query a particular body of text data. Consequently, match results arelikely to be less than comprehensive.

Obviously, full content data searching is better, but it is typicallycost prohibitive in prior art systems, because of the demands on systemresources. Therefore, there is a need in the art for an efficient fullcontent data searching technique. The technique should work withdisparate content data sources and disparate content data types. Thetechnique also should minimize search times by utilizing a build processto pre-process the full content data to streamline searching duringrun-time operation. The technique also should support natural wordsearch queries and should use alternative search words and word pairs toincrease the accuracy of search results and search process speed.

SUMMARY OF THE INVENTION

The present invention provides a full content data indexing system forprocessing content data associated with data applications such aselectronic encyclopedia and dictionary applications. A build processcollects content data from various sources, processes the content datainto constituent parts, including alternative word sets, and stores theconstituent parts in structured database tables. A nm-time process isused to query the database tables and the results in order to provideeffective matches in an efficient manner. Run-time processing isoptimized by preprocessing all query-independent steps during the buildprocess. A double word table representing all possible word paircombinations for each index entry and an alternative word table are usedto further optimize run-time processing.

The build process can break the content data down into words and tokenswith a Natural Language Parser (NLP) and apply an alternative word setto identify likely alternative search terms corresponding to the wordsand tokens. The build process stores the words and relationships in aset of database tables. The run-time process queries the databasetables, ranks the results, and returns the best matches.

The present invention can solve the above problems by providing a searchengine to better match user requests for information. The search engineallows users to search and retrieve information from a body of contentdata. It can provide users with general or specific queries to generalor specific content in the body of information. For example, users canbe directed to general information, such as the start of a long article,or to specific content within that article. An article outline andrelated articles also can be navigated. Queries can also be processed ina way that allows for quick results and an efficient use of systemresources.

In one aspect of the invention, a computer system is provided forsearching and retrieving information from at least one content sourcecontaining at least one content entity. The system includes a buildprocess for storing content information associated with the contententity in an index stored in the searchable content database. The systemalso includes a run-time process that can receive at least one searchterm and processes the search term against the index in the searchablecontent database. The build process also can create an alt word tableincluding at least one alternate word associated with the search term,so that the run-time process can identify a second match between thealternate word and the index and to return at least one search resultcorresponding to the second match.

In another aspect of the present invention, a method is provided forsearching and retrieving content from at least one content source. Themethod includes a step of building a search index table having indexentries corresponding to content information contained in the contentsource. The search index includes a double word table having at leastone word pair corresponding to the index entries. When a search term isreceived, the search term is processed against a portion of the searchindex table including a word pair corresponding to the search term todetermine whether a match is available. If a match is available, asearch result is returned identifying a content entity.

The various aspects of the present invention may be more clearlyunderstood and appreciated from a review of the following detaileddescription of the disclosed embodiments and by reference to thedrawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary operatingenvironment for implementation of various embodiments of the presentinvention.

FIG. 2 is a block diagram depicting the primary functional components ofan exemplary embodiment of the present invention.

FIG. 3 is a block diagram depicting an exemplary search index that maybe created as part of an exemplary build process.

FIG. 4 is a block diagram depicting the primary components of anexemplary rules table.

FIG. 5 is a block diagram depicting an exemplary search index table.

FIG. 6 is a block diagram depicting an exemplary search word table.

FIG. 7 is a flow chart depicting an overview of an exemplary buildprocess.

FIG. 8 is a flow chart depicting an exemplary method for performing aruntime process.

FIG. 9 is a flow chart depicting a detailed method for performing abuild process of an exemplary embodiment of the present invention.

FIG. 10 is a flow chart depicting a detailed run-time operation methodthat is an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Exemplary embodiments of the present invention provide a full textindexing system for processing content associated with data applicationssuch as encyclopedia and dictionary applications. A build processcollects data from various sources, processes the data into constituentparts, including alternative word sets, and stores the constituent partsin structured database tables. A run-time process is used to query thedatabase tables and the results in order to provide effective matches inan efficient manner. Run-time processing is optimized by preprocessingall query independent steps during the build process. A double wordtable representing all possible word pair combinations for each indexentry and an alternative word table are used to further optimizerun-time processing.

An Exemplary Operating Environment

Exemplary embodiments of the present invention will hereinafter bedescribed with reference to the drawings, in which like numeralsrepresent like elements throughout the several figures. FIG. 1illustrates an exemplary operating environment for implementation of thepresent invention. The exemplary operating environment includes ageneral-purpose computing device in the form of a conventional personalcomputer 120. Generally, the personal computer 120 includes a processingunit 121, a system memory 122, and a system bus 123 that couples varioussystem components including the system memory 122 to the processing unit121. The system bus 123 may be any of several types of bus structuresincluding a memory bus or memory controller, a peripheral bus, and alocal bus using any of a variety of bus architectures. The system memoryincludes a read-only memory (ROM) 124 and a random access memory (RAM)125. A basic input/output system (BIOS) 126, containing the basicroutines that help to transfer information between elements withinpersonal computer 120, such as during start-up, is stored in ROM 124.

Personal computer 120 further includes a hard disk drive 127 for readingfrom and writing to a hard disk, not shown, a magnetic disk drive 128for reading from or writing to a removable magnetic disk 129, and anoptical disk drive 130 for reading from or writing to a removableoptical disk 131 such as a CD-ROM or other optical media. Hard diskdrive 127, magnetic disk drive 128, and optical disk drive 130 arcconnected to system bus 123 by a hard disk drive interface 132, amagnetic disk drive interface 133, and an optical disk drive interface134, respectively. Although the exemplary environment described hereinemploys hard disk 127, removable magnetic disk 129, and removableoptical disk 131, it should be appreciated by those skilled in the artthat other types of computer readable media which can store data that isaccessible by a computer, such as magnetic cassettes, flash memorycards, digital video disks, Bernoulli cartridges, RAMs, ROMs, and thelike, may also be used in the exemplary operating environment. Thedrives and their associated computer readable media provide nonvolatilestorage of computer-executable instructions, data structures, programmodules, and other data for personal computer 120.

A number of program modules may be stored on hard disk 127, magneticdisk 129, optical disk 131, ROM 124, or RAM 125, including an operatingsystem 135, a data application 136, a search engine 138, and a database139. Program modules include routines, sub-routines, programs, objects,components, data structures, etc., which perform particular tasks orimplement particular abstract data types. Aspects of the presentinvention may be implemented in the form of a search engine 138 that canoperate in concert with the data application 136 and the database 139.The search engine 138 generally comprises computer-executableinstructions for binding and searching index tables. The database 139 isgenerally accessible to the search engine 138, but also can beimplemented as an integral part of the search engine.

A user may enter commands and information into personal computer 120through input devices, such as a keyboard 140 and a pointing device 142.Other input devices (not shown) may include a microphone, joystick, gamepad, satellite dish, scanner, or the like. These and other input devicesare often connected to processing unit 122 through a serial portinterface 146 that is coupled to the system bus 123, but may beconnected by other interfaces, such as a parallel port, game port, auniversal serial bus (USB), or the like. A display device 147 may alsobe connected to system bus 123 via an interface, such as a video adapter148. In addition to the monitor, personal computers typically includeother peripheral output devices (not shown), such as speakers andprinters.

The personal computer 120 may operate in a networked environment usinglogical connections to one or more remote computers 149. Remote computer149 may be another personal computer, a server, a client, a router, anetwork PC, a peer device, or other common network node. While a remotecomputer 149 typically includes many or all of the elements describedabove relative to the personal computer 120, only a memory storagedevice 150 has been illustrated in the figure. The logical connectionsdepicted in the figure include a local area network (LAN) 151 and a widearea network (WAN) 152. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets, and the Internet.

When used in a LAN networking environment, the personal computer 120 isoften connected to the local area network 151 through a networkinterface or adapter 153. When used in a WAN networking environment, thepersonal computer 120 typically includes a modem 154 or other means forestablishing communications over WAN 152, such as the Internet. Modem154, which may be internal or external, is connected to system bus 123via serial port interface 146. In a networked environment, programmodules depicted relative to personal computer 120, or portions thereof,may be stored in the remote memory storage device 150. It will heappreciated that the network connections shown are exemplary and othermeans of establishing a communications link between the computers may beused.

Moreover, those skilled in the art will appreciate that the presentinvention may be implemented in other computer system configurations,including hand-held devices, multiprocessor systems, microprocessorbased or programmable consumer electronics, network person computers,minicomputers, mainframe computers, and the like. The invention may alsobe practiced in distributed computing environments, where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

FIG. 2 is a block diagram depicting the primary functional components ofan exemplary embodiment of the present invention. In one embodiment ofthe present invention, a search engine 204 is used to process searchqueries generated as part of the normal operation of a data application200. The data application 200 may be, for example, an encyclopediaapplication program, a dictionary application program, or any other dataapplication that is associated with searchable data. Typically, a userof the data application 200 will generate a search query by entering anatural language query. In either case the search engine 204 can processthe query and search the data in a database 202 for one or more entriesmatching the query. Obviously, various tolerances may be applied to theidentification of a match, such that exact matches are returned as wellas matches that are less than exact.

For the purposes of a data application 200 such as an encyclopedia, thedatabase 202 may contain various kinds of entities such as articles,media, archive articles, audio files, video files, and index entries. Inaddition, the database 202 may include data associating one or moreentries with one another. For example, the database 202 may include datalinking an article with a side bar or with a related archive article.These associations may be represented by, for example, pointers, whichare a well-known means for representing relationships between data.

In one embodiment of the present invention, the entities that populatethe database 202 may be managed by a content management system 212.Content management systems are known in the art and are typically usedto manage the content of a website and other content-based applications.In an exemplary environment of the present invention, the search engine204 builds the database 202 during a build process. The search engine204 acquires content from the content management system 212 and builds asearch index in the database 202. The content from the contentmanagement system 212 is processed and organized by the search engine204 in accordance with rules that are stored in a rules table 210.Accordingly, the build process can be tailored to a particularapplication through the creation and modification of rules in the rulestable 210.

In addition to the build process, a run-time process is also supportedby an exemplary embodiment of the present invention. During run-time,the search engine 204 receives a query from the data application 200 andprocesses the query against the database 202. The search engine 204 mayuse a natural language parser 206 to process queries to optimize thesearch process. For example, if a query is entered as a natural languagesentence or phrase, the natural language parser may reduce it to a setof key words by eliminating unnecessary words from the query.

As stated above, the data client application 200 may be run on astand-alone computer or may be nm over a network, such as the internet.In either case, the runtime process should be optimized to return thebest search results in the least amount of time. This is especially truefor the on-line operation of the data application 200. On-line users ofdata client applications tend to be very sensitive to delays in theruntime process. Accordingly, the exemplary embodiments of the presentinvention are directed to optimizing run-time processing by implementinga novel build process that reduces the search time required to returnacceptable search results in response to a search query.

FIG. 3 is a block diagram depicting an exemplary search index 300 thatmay be created as part of an exemplary build process. The search engine204 may build a search index within the database 202 by processingcontent from the content management system 212 in accordance with rulesin the rules table 210. The content may include articles 304, archives306, media 308, audio and video files 310, and index entries 312. Thoseskilled in the art will appreciate that some content (e.g., audio files,media files) may be stored in a remote location outside the database byusing pointers to identify the remote storage location. Each entity 302may include one or more associated metadata items 314. Exemplarymetadata items include article titles 316, article word counts 318 andarticle categories 320. Those skilled in the art will appreciate thatthe entities 302 and the metadata 314 described above are provided onlyas examples and that the various embodiments of the present inventioncan be used to process various kinds of entities and metadata besidesthose specifically listed.

As stated above, the search engine 204 applies rules from a rules table210 to create the search index 300 within the database 202. FIG. 4 is ablock diagram depicting the primary components of an exemplary rulestable 210. The exemplary rules table 210 includes three sub-tables. Aclass table 402 includes entries corresponding to all of the entities inthe content management system 212—that will be affected by the buildprocess. The rules table 210 also includes an entity table 406 whichincludes all of the metadata associated with the entities in the contentmanagement system 212 that are affected by the build process. Aclass/entity table 404 represents the intersection between the classtable 402 and the entity table 406. In an exemplary embodiment, a searchsource table (not shown) may be used to group index entries into anappropriate rank, based on the search source with which the index entryis associated.

The rules tables 402, 404, 406 can be used to determine the structure ofthe search index 300. The rules tables 210 determine, for example, whichdata is indexed in the search index table, which data is availablefollowing the build process, and which data is processed by the buildprocess. Advantageously, the rules table 210 can be used by the searchengine 204 to create tables within the search index table 300 thatrepresent associations between content data, so that at run-time,queries can be processed more effectively and more efficiently.Specifically, exemplary embodiments of the present invention performsubstantially all non-query specific search operations during the buildprocess. Accordingly, the run-time process (i.e., the search process) isoptimized by the elimination of run-time operations.

One purpose of the build process is to process all content data intotables that can be more easily and efficiently queried during run-time.FIG. 5 is a block diagram depicting an exemplary search index table 300.As stated above, the search index table 300 is created as part of thebuild process to enable a more effective and efficient search operationduring the run-time process. In order to optimize the runtime process,the build process creates four tables 502-508 within the search indextable 300. The four tables are the search word table 502, the searchcontent word table 504, the search content table 506, and the searchcontent double word table 508.

The purpose of the build process is to populate these four tables foruse in processing search queries during subsequent run-time operation.The search content double word table 508 serves the purpose of storingword or token pairs that have been identified in the content data. Bystoring the double words, unnecessary search operations can be avoided.For example, where a search query includes the search terms “Russian”and “History”, the double word table 508 can be used to identify indexentries that include this word pair, thereby reducing the number ofentries that must be processed.

The search content table 506 contains a complete list of anything thatis indexed and/or searched on including, but not limited to, indexentries, titles, sentences, and section titles. The search word table502 contains a list of unique words, but does not include any stopwords. The search content word table 504 includes words that areattached to each entry in the search content table in a predefinedorder, but contains no duplicates. The search content double word tableis substantially identical to the search content word table, except thatit includes unordered, unique pairs within a single search content tableentry.

FIG. 6 is a block diagram depicting an exemplary search word table 502.The search word table includes three data types 602, 604, and 606. Analt words data type 602 contains alternative words that represent wordsthat are similar to one or more words or tokens in the search query. Altwords can include synonyms, common misspellings, and common phrasesassociated with the query terms. All of the words that are identified aspart of a set share an identical identification number (or other uniqueidentifier). An NLS tokens data type 604 contains tokens that may befound among query terms. The identification and processing of tokens canreduce search times by recognizing that the two or more words of thetoken should be processed as a single token. Finally, the search wordtable 502 includes a normal words data type 606. The normal words datatype simply contains all of the normal words that are contained in thecontent and that are not found in the other two data types.

FIG. 7 is a flow chart depicting an overview of an exemplary buildprocess. FIG. 7 begins at start block 700 and proceeds to step 702. Atstep 702, the searchable data is identified. This step can be performedby identifying a data source, such as a content management system. Theidentified data is searchable to the extent that the data can becompared to a query to produce a set of matches.

Once the data source has been identified, the method proceeds to step704. At step 704, the text data is divided into words and tokens. Asdescribed above, tokens are representations of words that are commonlyfound together and can include one or more words. Step 704 can beperformed by a word parsing module such as a natural language parser ornatural language system.

Once the text data has been divided into words and tokens, the methodproceeds to step 705. At step 705, all duplicates are removed, and themethod proceeds to step 706, wherein an alternative word set is applied.In step 706, alternative words associated with a word or token found inthe data, can be identified. Typically, alternative words consist ofsynonyms, common misspellings, and common related phrases. Whenalternative words are associated with a particular word or token, asubsequent search for that particular word or token can be made moreefficient. In short, alternative words are words that are expected to befound in a query directed to a target word which the alternative wordsare associated.

Once the alternative word sets have been applied, the target words andany relationships with alternative words are stored in a database atstep 708. This database can be implemented as the search index tabledescribed above. Once this database has been created, the build methodterminates by proceeding to end block 710. Accordingly, the buildprocess is terminated and the database has been prepared for searchingduring a run-time process.

FIG. 8 is a flow chart depicting an exemplary method for performing aruntime process. The method of FIG. 8 begins at start block 800 andproceeds to step 802. At step 802, a query is received. This query istypically received from a user of a data client application, such as anencyclopedia program. Those skilled in the art will appreciate that anyapplication for processing searchable data may serve as a source of sucha query.

The method proceeds from step 802 to step 804. At step 804, a databasequery is composed. As described above, the original query received maycontain natural word sentences or phrases or may contain other itemsthat can hamper the search process. At step 804, the query is processedto make the query conducive to the known architecture of the database.

Once the database query has been composed in step 804, the methodproceeds to step 806. At step 806, the database is queried. In short,the database query is compared to the database to generate a list ofpotential matches or results. As stated above, the database that isqueried in step 806 could be a search index table.

The method proceeds from step 806 to step 808 and the results areranked. The purpose of ranking the results is to provide the searchresults in descending order, based on a calculated likelihood that aparticular result entry is a target of the search query.

The method proceeds from step 808 to step 809 and any duplicates areremoved. The method proceeds from step 809 to step 810 and the bestmatches (i.e., those with the highest ranking) are returned. Thoseskilled in the art will appreciate that various threshold levels couldbe set to determine which results are returned. The method proceeds fromstep 810 to end block 812 and terminates.

FIG. 9 is a flow chart depicting a detailed method for performing abuild process of an exemplary embodiment of the present invention. Themethod begins at start block 900 and proceeds to step 902. At step 902,a skeleton database (i.e., empty database) is created to store thesearch index table and all supporting tables. The method proceeds fromstep 902 to step 904. At step 904, the supporting tables, including theclass table, the class/entity table, and the entity table, are copiedinto the skeleton database. As stated above, these tables include therules of how to process the content data.

The method proceeds from step 904 to step 906. At step 906, index datais inserted into the empty tables in the database. Index data includeindex entries that are essentially pointers to content. After the indexentries have been inserted, the method proceeds from step 906 to step908. At step 908, the encyclopedia entities are processed. In step 908,entities associated with the content of the encyclopedia dataapplication are inserted in the search index. As stated above theseentities include metadata associated with specific content. Suchmetadata can include article titles, word counts, and articlecategories. After the encyclopedia entities have been processed, themethod proceeds from step 908 to step 910.

At step 910, the encyclopedia text is processed. In short, this stepinvolves adding to the database text that corresponds to the entitiesprocessed in step 908. The method proceeds from step 910 to step 912. Atstep 912, forward associations are processed to associate entities inthe database. For example, an encyclopedia article may be associatedwith media, a web link, or an archived article. These associations areestablished within the database so that the content will be properlyassociated at run-time.

The method proceeds from step 912 to step 914. At step 914, reverseassociations are processed for the encyclopedia content. Reverseassociations are helpful in cases where, for example, a search resultmay include a narrow content entity, but should also include the broadercontent entity that contains the narrow content entity. The reverseassociation process will establish a link that enables the search forinclude such flexibility. Those skilled in the art will appreciate thatwhile there may be some overlap between forward associations and reverseassociations, they are not necessarily mutually inclusive.

The method proceeds from step 914 to step 916. At step 916, compound andcomposite media are processed. Compound media are content entities thatinclude more than one content entity, such as a picture and anassociated audio file. Composite media are content entities that mayinclude simple and compound content entities. The method proceeds fromstep 916 to step 918.

At step 918, dictionary data (as opposed to encyclopedia data) isprocessed in the same manner as described in connection with steps908-916. In this embodiment of the present invention, processing isprovided for an encyclopedia data application as well as for adictionary data application. Those skilled in the art will appreciatethat exemplary embodiments of the present invention may be used inconjunction with one or more data applications. The method proceeds fromstep 918 to step 920. At step 920, related articles are processed.Related articles are hierarchical lists of content related to aparticular article.

The method of FIG. 9 proceeds from step 920 to step 922. At step 922,index data is processed. In this step, the index data inserted in step906 is stored in the database. In one embodiment of the presentinvention the index key words are stored in one or more of the fourtables described in more detail in connection with FIG. 5. The methodproceeds from step 922 to step 924. At step 924, content browse data isprocessed. This step essentially sorts content that belong to identifiedcategories. At run-time, a user may browse the sorted articles based onan identified area of interest and/or category.

The method proceeds from step 924 to step 928. At step 928, the methodprepares for run-time operation. In this step, the populated tables canbe cleaned up so that unnecessary entries in the tables are removed. Themethod proceeds from step 928 to step 930. At step 930, the output fileincluding the search index table and all other populated tables can bedetached from the build server. The method proceeds from step 930 to endblock 932 and terminates.

FIG. 10 is a flow chart depicting a detailed run-time operation methodthat is an exemplary embodiment of the present invention. The method ofFIG. 10 begins at start block 1000 and proceeds to step 1001. At step1001, a user query is received. The method proceeds from step 1001 tostep 1002 and the query is transmitted for processing. In the embodimentdepicted in FIG. 10, the query is sent to a web service which can be anapplication executed on a network-based machine. A web service is amodule of application logic that can be made accessible to otherfunctional modules by way of standard network (e.g., internet)protocols. Advantageously, this method of accessibility can usually beaccomplished such that the web service can be provided in aplatform-independent manner.

The method proceeds from step 1002 to step 1003. At step 1003, the queryis converted to tokens. The method proceeds from step 1003 to step 1004and the original user query and the tokenized query are sent to adatabase. The method proceeds to step 1005, wherein an exact match isconducted on the user query. The method proceeds from step 1005 to step1006 and an exact match is conducted on the first token.

The method proceeds from step 1006 to step 1007. At step 1007, each wordin the query is looked up in the search word table and all valid wordsare identified. The method then proceeds to step 1008 and word pairs arecreated from identified valid words. The method proceeds from step 1008to step 1009. At step 1009, the search content double word table issearched using the word pairs created in step 1008. The method thenproceeds to step 1010 and the search content word table is searchedusing the original words. The method then proceeds to step 1011 and allresult sets produced by steps 1009 and 1010 are returned. In theembodiment of FIG. 10, these result sets are returned to the webservice.

The method proceeds from step 1011 to step 1012, wherein the results aremerged into a single list and duplicates are removed. The method thenproceeds to step 1013 where the results list is converted to XML. Themethod proceeds from step 1013 to step 1014 and the XML-based resultslist is returned. The method then proceeds to end block 1015 andterminates.

Although the present invention has been described in connection withvarious exemplary embodiments, those of ordinary skill in the art willunderstand that many modifications can be made thereto within the scopeof the claims that follow. Accordingly, it is not intended that thescope of the invention in any way be limited by the above description,but instead be determined entirely by reference to the claims thatfollow.

1. In a computing system having access to multiple content entities, each content entity including searchable content, a method for building a database for facilitating searching and retrieving of content entities in an efficient manner that returns results of content entities expected to be found, the method comprising: creating a skeleton database for storing a search index table and one or more other tables for facilitating a search for content entities within one or more content sources; inserting index data into the skeleton database, the index data including index entries pointing to content within the one or more content sources; processing content entities from a first content source and inserting data associated with the content entities into the search index table; adding to the skeleton database associations between content entities of the one or more content sources and processing related content entities identified by the associations; and outputting the skeleton database into an output file that includes the search index table and the one or more other tables, and detaching the output file from a build server used to create the skeleton database.
 2. A method as recited in claim 1, further comprising: adding to the skeleton database the one or more other tables, wherein the one or more other tables include rules on how to process the content within the one or more content sources.
 3. A method as recited in claim 2, wherein the one or more other tables includes a class table.
 4. A method as recited in claim 2, wherein the one or more other tables includes a class/entity table.
 5. A method as recited in claim 2, wherein the one or more other tables includes a entity table.
 6. A method as recited in claim 1, wherein adding to the skeleton database associations between content entities of the one or more content sources includes adding forward associations.
 7. A method as recited in claim 1, wherein adding to the skeleton database associations between content entities of the one or more content sources includes adding forward associations between content entities.
 8. A method as recited in claim 1, wherein adding to the skeleton database associations between content entities of the one or more content sources includes adding backward associations between content entities.
 9. A method as recited in claim 1, further comprising adding text to the database, the text corresponding to the processed content entities.
 10. A method as recited in claim 1, further comprising processing related content entities, the related content entities being included in hierarchical lists of content related to a particular content entity.
 11. A method as recited in claim 1, further comprising storing index key words in the search index table and one or more other tables, and removing unnecessary entries in the tables.
 12. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause a computing system to perform the method of claim
 10. 13. In a computing system having access to multiple content entities, each content entity including searchable content, a method performing a run-time search for tokens related to key terms input in a user query, the method comprising: receiving a user query, the user query including one or more key terms input by a user for searching and retrieving content entities; converting the user query to a tokenized query; sending the original user query and the tokenized query to a database; in the database, conducting an exact match on the user query; in the database, conducting an exact match on the tokenized query; identifying all terms in the user query and tokenized query which are valid words in the one or more search indexes; searching the one or more search indexes with the key terms of the user query; and returning to a user a result set of all content entities matching the user query and the tokenized query.
 14. A method as recited in claim 13, further comprising sending the user query to a web service.
 15. A method as recited in claim 13, wherein identifying all key terms in the user query and tokenized query which are valid words in the one or more search indexes includes looking up each key term and token in a search word table.
 16. A method as recited in claim 13, further comprising creating word pairs from all valid words, and searching a search content double word table with the word pairs.
 17. A method as recited in claim 16, further comprising searching a search content word table with the key terms.
 18. A method as recited in claim 13, wherein returning to a user a result set of all content entities matching the user query and the tokenized query comprises returning a plurality of result sets to a web service.
 19. A method as recited in claim 18, further comprising merging the plurality of result sets into a single list.
 20. A method as recited in claim 19, further comprising removing all duplicate content entities from the single list. 